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Abstract 

In this paper, we consider orthogonal multiple access coded schemes with correlated sources. The 
sources are distributedly encoded and their correlation is exploited at the access point (AP). In particular, 
we assume that each source uses a classical channel code to transmit, through an additive white Gaussian 
noise (AWGN) channel, its information to the AP. Here component decoders, associated with the source 
encoders, iteratively exchange soft information by taking into account the source correlation. The key 
goal of this paper is to investigate the ultimate achievable performance limits in terms of a multi- 
dimensional feasible region in the space of channel parameters, deriving insights on the impact of 
the number of sources. In order to analyze the performance of given coded schemes, we propose an 
extrinsic information transfer (EXIT)-based approach, which allows to determine the corresponding 
multi-dimensional feasible regions. On the basis of the proposed analytical framework, the performance 
of pragmatic schemes, based on serially concatenated convolutional codes (SCCCs) and low-density 
parity-check (LDPC) codes, is discussed. 

Index Terms 

Correlated sources, orthogonal multiple access, joint channel decoding (JCD), noisy Slepian-Wolf 
problem, EXIT chart. 
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I. Introduction 

The efficient transmission of correlated signals, observed at different nodes, to one or more 
collectors is one of the main challenges in various networking scenarios, e.g., wireless sensor 
networks [1]. In the case of one collector node, this problem is often referred to as reach-back 
channel problem [2]-[4]. In its simplest form, it can be summarized as follows: two independent 
nodes have to transmit correlated sensed data to a collector node by using the minimum possible 
energy, e.g., by properly exploiting the implicit correlation among the data. 

In the case of separated additive white Gaussian noise (AWGN) channels, the separation 
between source (up to the Slepian-Wolf limit) and channel coding is known to be optimal [2], 
[5]. However, implementing a practical system based on separation, i.e., distributed source 
coding (DSC) followed by channel encoding, is not straightforward [6]-[9] and the design of 
practically good codes is still an open issue [10]. Alternative approaches are given by cooperative 
source-channel coding and distributed joint source-channel coding (JSCC). In the JSCC case, no 
cooperation among nodes is required, each correlated source is encoded by a single code, and 
the correlation between the sources is exploited at the joint decoder by means of joint channel 
decoding (JCD) [1 1]— [16]. In other words, for a given source neither the realization from the 
other sources nor the correlation model are available at the encoder. Correlation between the 
sources must instead be assumed to be known at the (common) receiver, which aims at the 
reconstruction of the information streams transmitted by the sources. 

Work dealing with JCD has so far considered classical concatenated or low-density parity- 
check (LDPC) coded schemes, where the decoder can exploit the correlation among the sources 
by performing message passing between the corresponding subdecoders. Although a significant 
attention has been recently paid to these topics, the problem of designing good codes has been 
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only partially addressed, mainly with concatenated codes. In [16], the authors state that for two 
separate channels — which is a case of interest in this paper — the type of concatenated code 
utilized for the encoding is not critical, and good results can be obtained, provided that powerful 
codes are employed. In [17], recursive nonsystematic convolutional encoders are proposed as 
constituent encoders for heavily biased sources, leading to a signal-to-noise ratio (SNR) penalty 
between 0.74 dB and 1.17 dB from the Shannon limit. Other approaches to code design for 
scenarios with two correlated sources proposed in the literature can be mentioned. In [18], 
optimized LDPC codes are designed, by means of puncturing and proper iterative decoding 
schedule at the AP. Extensions to universal codes, i.e., capacity-achieving codes for all possible 
channel parameters, through spatial coupling is also presented in [19], [20]. 

In this paper, we extend our previous work, appeared in preliminary form for two-source 
scenarios in [21], [22], considering a scenario with a generic number of correlated sources which 
transmit to a common access point (AP) through orthogonal AWGN channels. The sources do not 
explicitly use source codes, but only standard channel codes. At the AP, proper iterative receivers 
are used to exploit the source correlation. This scenario has been also considered in [23], where 
practical coding/decoding schemes have been designed in the presence of block faded channels. 
In this work, we analyze the multi-dimensional feasible region in the space of channel parameters 
characterizing an arbitrary number of sources, where error-free communication is allowed. It will 
be shown that a few characteristic points are sufficient to accurately characterize this feasible 
region. The limiting behavior, for large values of the number of sources, is investigated. The 
impacts of the correlation and of the number of sources are also discussed. We then apply our 
framework to two families of channel codes: serially concatenated convolutional codes (SCCCs) 
and LDPC codes. By using density evolution, we propose an operational extrinsic information 
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transfer (EXIT)-inspired approach to evaluate the performance, in terms of feasible region, of 
the considered practical channel coded schemes with iterative decoding at the AP. We remark 
that our approach leads naturally to optimized channel code design. 

This paper is structured as follows. In Section [III preliminaries on the scenario of interest 
are given. In Section [nil the multi-dimensional feasible region is introduced together with its 
information-theoretic characterization for large number of sources. In Section [IV] the principle 
of JCD is concisely recalled. Performance results (for SCCCs and LDPC codes), based on an 
EXIT approach described in the Appendix, are presented and discussed in Section |V] Finally, 
Section [VI] concludes the paper. 

II. Scenario 

Consider n spatially distributed sensor nodes which detect (i.e., receive at their inputs) binary 
information sequences = (x\ , . . . ,x]//), where k = 1,. .. ,n denotes the sensor index and L 
is the signal length assumed equal for all sources. The information signals are assumed to be 
independent with P(xf = 0) = P{xf^ = 1) = 0.5 and the following simple correlation model is 
considered: 

xf =bi®z\ k) i = l,...,L k=l,...,n (1) 

where {bi} are independent and identically distributed (i.i.d.) binary random variables and {z\ } 
are i.i.d. binary random variables with P(z\ = 0) = p, with 1/2 < p < 1. This correlation model 
corresponds to a scenario where the sources sense the output of a set of binary symmetric 
channels (BSCs), with cross-over probability p, whose input is a common information bit b\. 
Obviously, if p = 0.5 there is no correlation between the binary information signals {x^}1 =l , 
whereas if p = 1 the information signals are identical with probability 1 . According to the chosen 
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correlation model, the a priori joint probability mass function (pmf) of the information signals 
at the inputs of the n sources at the j-th epoch, i 6 {1, . . . ,L} can be computed. After standard 
manipulations, one can show that 



P(*i) = E P(xi\bi)p(bi 

1 



&i=0,l 



[p llh (l-p) n - nh + (l-p) nb p n - nb ] i=l,...,L (2) 

where Xj = (x\ l \ . .. ,x- ) r is the column vectoi^ denoting the bits at the input of the various 
sensors at time epoch i and = n^Xj) is the number of zeros in Xj. 

In Fig. Q3 the overall model for the multiple access scheme of interest is shown: n source nodes 
communicate directly (and independently of each other) to the AP. The information sequence 
at the k-th source node is encoded using a binary linear code, denoted as ^ (k = 1 , . . . , n) with 
codewords {s^}^ =l , s\ k ' E {0, 1}, of length assumed equal for all source nodes. Therefore, the 
encoding rate at each source is r = L/N. The goal of the communication system is recovering, at 
the AP, the information signals {jc^}t =1 with arbitrarily small probability of error. Assuming that 
binary phase shift keying (BPSK) is the modulation format, after matched filtering and carrier- 
phase estimation, the real observable at the AP, relative to a transmitted binary information 
symbol, can be expressed as 



y m = v p + n v> = ^) ^ i = h N k=h n (3) 

(k) (k) (k) 

where v ; - denote the antipodal transmitted BPSK signals with energy Eq and J]- ' are AWGN 
independent random variables with zero mean and variance Nq/2. 

'The notation (-) T denotes the transpose operator. 
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Fig. 1. Proposed multiple access communication scenario: n source nodes communicate directly to the AP. 



For conciseness, the following matrices can be defined: 

X = (xi,X2,...,Xl) 

S 4 



(si,s 2 ,...,s N ) = ((s [l) ) , (• 
(yi,yi,---,yN) = > (y 



In other words, X is an h x L matrix whose rows are the information bits at each source. 
Similarly, S is an nx N matrix whose rows are the codewords transmitted by each source node 
and Y is an n x N matrix whose rows are the received vectors at the output of each of the n 
orthogonal channels. In the following sections, the notation p(A) denotes the joint probability 
density function (pdf) of the elements of matrix A. 
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III. Feasible Region 

In the described scenario, the performance achievable by a DSC scheme followed by channel 
coding is identical to that achievable if the sources were jointly channel encoded. The Slepian- 
Wolf (SW) theorem allows to determine the achievable rate region for the case of separate lossless 
encoding of correlated sources. Denoting by r s ^ the source encoding rate for k-th transmitter, 
the SW region [24] can be compactly formulated as the intersection of the family of inequalities 

Y,r Sjkm >H(n)-H(n-p) (4) 

where pe{l,...,n}, {k\, . . .,k p } C {l,...,n}, and 

= -\ i ( n ) [p Mb (i-pr nb +(i-p) nb P n ~ nb ] 

•iog 2 [p ,ib (i -pr nb + (i -py h P n - nh ] } (5) 

with the conventional assumption that H(0) = 0. The formulation © can be derived by straight- 
forward manipulations and can be found, e.g., in [23]. By assuming that source coding is followed 
by channel coding, the channel code rates {r c ^}'l=i ma y be expressed as 

r c ,k = r S) k ■ r (6) 
where we recall that r = L/N. The channel code rates must satisfy the following Shannon bounds: 

r C) *<4 = ~log 2 (l + %) k=l,...,n (7) 

where and are the capacity, in bits per channel use, and the SNR at the AP relative 
to the &-th link, respectively. As anticipated in Section HI compressing each source up to the 
SW limit and then utilizing independent capacity-achieving channel codes allows to achieve the 
ultimate performance limits [2], [5]. Combining ©, ©, and ©, a feasible region of individual 
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capacity values characterizing the set of orthogonal channels can be identified by the following 
inequalities to be jointly satisfied by the link capacities {^k}1=v 

i X km >r[H{n)-H{n-p)} (8) 

m=l 

for pe{l,...,n| and {k\ , . . . , k p } C { 1 , . . . , n}. For brevity, we refer to this region as feasible 
capacity regions 

In Fig. [21 the feasible capacity region is shown in scenarios with (a) n = 2 sources and (b) 
n = 3 sources. In all cases, p = 0.95 and r = 1/2 at each source. In general, the border of the 
feasible capacity region is given by the intersection of the 2" — 1 hyperplanes defined by ©. 

2 We remark that our definition of feasible capacity region must not be confused with the definition of capacity of a multiple 
access channel. In our scenario, in fact, the capacity region refers to a set of capacity values of the individual links for which 
it is possible, for a given overall rate r, to achieve communication with arbitrarily small probability of error. 
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A. Characterization 

A few characteristic points can be easily identified on the border of the feasible capacity region. 
In particular, two characteristic operational regions, denoted as "balanced" and "unbalanced," can 
be identified. In the balanced case, the characteristic point on the border of the feasible region 
corresponds to a scenario where all sources are transmitted at the same single channel capacity, 
i.e., X\ = X% = ■ ■ • = X„. This value, denoted as Abai, can be determined by considering the 
hyperplane associated with p = n in ([8]), thus obtaining 

£ A,- = nA ba i = rH (n) 



i=l 

and, therefore, 



Abai = r . (9) 



In the unbalanced case, instead, the considered scenario is characterized by n — 1 sources, e.g., 
sources from 1 to n—\, with values of A; (i = l,...,n— 1) sufficiently large to satisfy the 
corresponding constraints of type ®. In this case, Au n b is the smallest value of X n such that the 
operational point lies on the border of the feasible region. This corresponds to considering the 
hyperplane associated with p = 1 and k\ = n in ©, thus obtaining 

X n = r[H(n)-H(n-l)]±X mh . (10) 

Note that Abai and A un b are functions of n, i.e., Ab a i = Abai(n) and A un b = A un b(n): for the sake 
of readability, we will not explicitly indicate the dependence on n — the context will eliminate 
any ambiguity. In Fig. [2l the characteristic (balanced and unbalanced) points on the border of 
the feasible capacity region are shown. 

We now investigate the behavior of the characteristic points Ab a i and X^^. To this end, the 
information sequence at the k-ih. sensor and z'-th epoch can be viewed as a stationary stochastic 
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process in the index k (for fixed i). In the remainder of this section, we adopt the notation 
(xi)1 = (x^ l \...,x^). Similarly, a portion of the process from index 1 to k will be denoted as 
(xi)\. For simplicity, we omit the subscript i denoting the time epoch at which the process is 
observed. With a slight abuse of notation, (xi)\ will be denoted as x\, whereas the single bit xf^ 
will be denoted as x k . Moreover, note that H{x t {) = H(n). 

Theorem 1 The characteristic points Abai and A un b are non increasing functions of n and, for 
every value of n, Abai > Kmb- 

Proof: By using the chain rule for the entropy [24], it follows that 

H(n) H{x\) 1 » k _ x 

4al = = r l - = r- > H(x k \x\ 1 . 

n n 

Since the process {x k } is stationary and H(xk\x\~ 1 ) is a non increasing function of k 

H(x k \^ l )>H(x n \x\- 1 ) i=l,...,Jfc 

hence 

1 ^ 



Therefore, one can write 



n k=\ n i=l 



Abal = r- f^H(x k \x\- 1 ) > rH(x n \x1- 1 ) = r[H(n) -H{n-\)\ 
n k=l 

where the last equality holds because 

H(n)-H(n-l)=H(x n l )-H(x n l - 1 )=H(x n \x n l - 1 ). 

Since r [H(n) — H(n — 1)] = Au nb , it is finally proved that Ab a i > r[H (n) — H (n — 1)] = A^b- 
Moreover, it is straightforward to observe that, by definition, Ab a i and A un b are non increasing 
functions of n. ■ 
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Theorem 2 For asymptotically large number of sources 



lim A unb = lim A ba i = t3HP = X hm 



where is the entropy rate of the stochastic process {x^}. 
Proof: From (flOl) . one can write 



lim Aunb = lim r [H(n) — H(n— 1)] . 

n— n— >+oo 

For a stationary stochastic process, the following limit equality holds [24] 



lim #(x w |jc? ) = lim 
Therefore, one obtains 

lim A un b = lim rHixAx^ 1 ) = lim r — = lim r = lim Afo a i- 

;i^+o° n— !>+o<3 «— !>+oo n »—►+<» n n—¥+°° 

Finally, note that this limit is equal to rJf 7 [24]. ■ 
Theorem [Hand [2] give insights on the characteristic points of the feasible capacity region. One 
may therefore investigate what is the limiting behavior of the border of the capacity region, i.e., 
how the shape of this n-dimensional hypersurface modifies for increasing values of the number 
of sources. 

Theorem 3 The following bounds on the joint entropy of the information sequence hold: 

1 + {n- l)Hb(p) < H[n) < 1 +nH h {p) 
where #b(p) is the entropy of a binary random variable with parameter p, i.e., 

H h (p) 4 -plog 2 p - (1 -p)log 2 (l -p). 
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Proof: Let us consider the model (Q]) of the source bit sequence x" based on the common 
bit b, where the index i has been omitted. The joint entropy of x\ and b, denoted as Hib.x"), 
can be expressed as 

H(b,xfl)=H(b)+H(x n 1 \b). (11) 

Since b is a uniformly distributed binary random variable, it follows that H(b) = £1^(0.5) = 1. 
Moreover, due to the correlation model (OQ), it is also possible to obtain 



H(x n l \b)=H(z n l )=nH h (p) 



where z" is defined, adopting a notation similar to x", for the random variables {z^ }. Therefore, 
one obtains: 

H(b,4) = l+nH h (j)). 
As in CCD), it is also possible to write 

l+nH b (p) = H(b,j%) = H{x\) +H{b\x\) 

and, therefore, 

H(n) = H(x1) = l+nH h (p) -H{b\x\). (12) 
Since by definition H(b\x") > 0, it follows that 

H(n)<l+nH h (p). (13) 
Moreover, since conditioning reduces entropy [24], it also follows that 

H(b\4)<H(b\ Xl )=H(zi)=H h (p). 
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Using this in (fT2j) . one obtains: 

H{n) = l+nH h (p)-H(b\x1) 

> l+nH b (p)-H b (p) 

= l + (n-\)H b (p). (14) 

Combining (fl"3T) and (fl4l) concludes the proof. ■ 
In Fig. [3l the sequence entropy is shown, as a function of the number of sources, in a scenario 
with r = 1/2 and three different values for p: (i) 0.9, (ii) 0.95, and (iii) 0.99. For each value of 
p, the lower and the upper bounds in Theorem [3] are also shown. As one can see, the upper and 
the lower bounds are quite close. Moreover, for sufficiently large values of n (namely, n > 6), 
H(n) approaches the linear function of n equal to the upper bound 1 + nH b (p). In order to better 
characterize this asymptotic convergence, the following considerations can be useful. Owing to 
the correlation model ((U), assume that an estimate b n of b can be constructed from x" by simply 
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considering a majority logic decision rule. As < 1 — p < 1/2, it follows that 

lim p(b n = b)=l. (15) 

n— V / 

In fact, for a sufficiently large value of n, with high probability more than half of the "obser- 
vations" of b (i.e., x") are equal to b itself and, therefore, the common source can be perfectly 
estimated. The following lemma crystallizes this behavior. 

Lemma 1 The following limit holds: 

lim {H(n)-[l+nH h (p)]} = 0. 

Proof: Since the knowledge of an estimate of b based on x'\ does not modify the entropy, 
conditionally on it holds that 

H(b\x n 1 )=H(b\x n l ,b n )- 

Using the estimate with the limiting property (fl"5l) . the uncertainty on b given its "perfect" 
estimate goes to zero for sufficiently large values of n, i.e., 

lim H(b\x n l )= lim #(&|jc?,S„) = 0. 

From the proof of Theorem [3l one finally obtains: 

lim H(n)= lim 1 +nH h (p) -H(b\xf) = lim [l+nH h (p)]. 

■ 

We now present another lemma useful to prove a following theorem which characterizes the 
asymptotic (i.e., for sufficiently large number of sources) shape of the feasible capacity region. 

Lemma 2 The asymptotic value X\{ m defined in Theorem [2] can be expressed as 

Ai im = rH b (p) 
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Fig. 4. Abai, A^b, and Aii m , as functions of n, in a scenario with r= 1/2. Three different values for p are considered: (i) 0.9, 
(ii) 0.95, and (iii) 0.99. 



Proof: From Theorem [3l one can write 

-+«b(p) 
n 

and, therefore, 

fi(«) , s 

lim r — ^ = rfi b (p). 

■ 

In Fig. |H Abai, Aunb, and X\ im are shown, as functions of n, in a scenario with r = 1/2 and three 
different values of p: (i) 0.9, (ii) 0.95, and (iii) 0.99. First, one can observe that, for increasing 
values of n, Ab a i and Au„b are decreasing functions of n, as predicted by Theorem [Q Therefore, 
the projection of the border of the feasible capacity region on a two-dimensional plane (e.g., the 
(Ai, A2) plane) enlarges with n. Moreover, it is also verified that Ab a i > A un b Vn, as predicted by 
Theorem Q3 and that both Ab a i and A^b approach the same limiting value X\\ m , as predicted by 



1 n-\ , . 

- + fib P 

n n 



H(n) 
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Theorem |3l We can also observe that in the unbalanced case the convergence is faster. Therefore, 
the shape of the feasible region tends to be a hyperoctant, as will be shown more clearly in the 
following theorem. 

Theorem 4 For large values of n, the border of the feasible region approaches a hyperoctant 
with the same limiting value, equal to Au m = rJ%?, for each dimension. In other words, Ve > 0, 3 
no such that Wn > no the n-dimensional vector ( Ai,m + £,... , An m + e) belongs to the feasible 
region. 

Proof: Since we want to prove that the vector ( An m + £,..., An m + e) belongs to the feasible 

region, the set of equations © have to be satisfied for all values of p: 

p p 

T,hn= L (hm + £) = (hm + £)p>r[H(n)-H(n-p)} p = l,...,n. 

m=l m=l 

Using Lemma [2] and (fT2l) . the following equalities hold: 

p(hm + £) = p(rH h (p)+e) 

= n(rH b (p) + £)-(n-p)(rH b (p)+e) 

= nrH h (p) - (n - p)rH h (p) + p£ 

= r + nrH h (p)-r-(n-p)rH b (p)+pe 

= r + nrH b (p)-rH(b\x'l)-r-(n-p)rH h (p) + rH(b\x n - p ) 

+pe + rH{b\4) - rH{b\x[~ p ) 
= rH(n) -rH(n-p)+pe + rH(b\tf) - rH(b\x n ~ p ). 
From Lemma \T\ for some value of n one can choose, as long as p > 0, a sufficiently small 8 



such that 



H{b\x\)-H{b\x n ~ p ) < H{b\x\)-H{b\x 



< 8 <pe. 
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The term pe + rH(b\x^) —rH(b\x\~ p ) is greater than or equal to zero, and, therefore, 

p(X ]im + e)>r[H(n)-H(n-p)) 
which finally concludes the proof. ■ 

Theorem 5 If one of the n channels operates in the unbalanced regime, then the capacities of 
the remaining n—1 channels must belong to the feasible region associated with n—\ sources. 

Proof: Let us consider a particular value of n and denote A„ = A un b(«)- The set of inequalities 
© identifying the feasible capacity region can be formulated as 

£ X km >mzx{r[H(n)-H(n-p)],r[H(n-\)-H{n-p-\)]} (16) 

m=l 

for q G {1, . . . ,n — 1} and {k\ , . . . , k q } C {1, . . . ,n — 1}. In fact, for a fixed value of q, {A^ , . . . , A^ } 
must satisfy the same inequalities in © with p = q and an additional inequality with p = q + 1 
and X n = Aunb = —H(n — 1)]. For any value of g, it can be easily proved that (fT6l) can be 

rewritten as 

9 [ p p 

^k m > max<^ J^AunbCn-i + l), J^Aurib(/»-0 

m=l l ! ~ 1 != 1 

Since {A un b(n)} is a decreasing function of n (see Theorem d]), one obtains 

q p 

m= 1 != 1 

= r[H(n-l)-H(n-l-p)} 
which are the set of inequalities identifying the feasible region with n — l sources. ■ 

Corollary 1 For n > 3, the (X\ , A2) projection of the border of the feasible capacity region 
in correspondence with the minimum possible values of A3 , . . . , X n equals to the border of the 
feasible capacity with n = 2. 
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Fig. 5. Projections of the ^-dimensional feasible region, in a scenario with r = 1/2 and p = 0.95, for n = 2 and n = 3. 

Proof: Let us first consider n = 3 and A3 = Au n b(3) (i.e., the minimum possible). From 
Theorem [5l the (Ai, A2) projection of the border of the feasible capacity region equals the border 
of the feasible capacity region for n = 2. Therefore, the corollary is verified. 

For larger values of n, the corollary can be proved by induction, by observing that from 
Theorem [5] the dimensionality of the problem can be reduced to n — 1. It is then sufficient to fix 
the remaining values of A3,...,A, t _i to the minimum possible values according to the fact that 

Arc = Aunb(rc)- ■ 

In Fig. [51 the (Ai, A2) projection of the border of the feasible capacity region with n = 2 is 
compared to those associated with n = 3 and various values of A3. In all cases, r = 1/2 and 
p = 0.95. Observe preliminary from Fig. [5] (a) that, according to Corollary [Q the border of the 
projection of the feasible capacity region with n = 3 and A3 = Au n b(3) corresponds to the border 
of the feasible region for n = 2. The projection of the feasible region with n = 3, for A3 > r, is 
shown in Fig. [5] (c). For intermediate values of A3, i.e., Au n b(3) < A3 < r, the projection of the 
feasible region with n = 3 is between the two previous cases: an illustrative example is shown 
in Fig. \5\ (b). 

In Fig. [61 we show the widest (Ai,A2) projections of the feasible capacity region for various 
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Fig. 6. Limiting curves of the projections of the rc-dimensional feasible region, in a scenario with r= 1/2 and p =0.95, for 
various values of n (2,3,4,oo). 

values of n: 2, 3, 4, and °°. One should observe that the projection of the feasible capacity region 
is very close to the limiting capacity region predicted by our analytical framework, for n > 3. 
This will be confirmed also for practical coding schemes, whose performance are characterized 
in more detail in Section |V] by computing their characteristic points Abai and A un b- 

B. Discussion 

From the characterization of the feasible capacity region carried out in the previous subsection, 
the following major conclusions can be drawn. 

As can be seen from Fig. [51 increasing the number of sources beyond 3 does not lead to 
significant performance improvement, as the projection of the feasible capacity region becomes 
only slightly larger. This means that a good channel code designed for a scenario with n = 2 
sources, i.e., with performance close to the theoretical limit in this setting, will likely be a good 
code also in scenarios with larger values of n. 

As can be seen from Fig. |4]and[5l when the number of sources is sufficiently large (e.g., n > 6), 
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the feasible capacity region basically coincides with the limiting (n = °°) hyperoctant and can 
be completely characterized by the parameter X\\ m = A^b, which is simpler to be characterized 
than Abai, as will be shown in Section |V] When the number of sources is smaller, instead, an 
accurate characterization of the feasible capacity region would require the identification of a few 
points in the ^-dimensional space (see Fig. [2] (b)). 

As the ultimate performance limits (in terms of feasible capacity region) have been char- 
acterized, it remains of interest to understand how given (pragmatic) channel coding schemes 
perform. To this end, in the remainder of this paper we generalize the EXIT chart-based method, 
introduced in [21], [22], to the performance analysis of channel codes in the multiple access 
scenario of interest. 

IV. JCD Principle 

Using the matrix notation introduced in Section [nj the joint maximum a posteriori probability 
(MAP) decoding rule, given that Y is received, reads: 

Jcf°= argmax £ p(Y\X)p(X) (17) 

(k) 

where i = 1 , . . . , L, k = 1 , . . . , n, and the notation X ~ x\ denotes that the summation runs over 
all variables in X except x\ . From (fTTT ), using the Bayes rule one can write: 

xf ] = argmax £ p(Y\X)p(X) 

= argmax £ p(Y\S,X)p(S\X)p(X) 

= argmax £ p (Y \S) p (S\X)p (X) (18) 

where the last line uses the fact that the joint pdf of Y, conditionally on S, does not depend 
onX. The probability p(S\X) is equal to one if jt 1 ',...,^ are the codewords associated with 
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Jt^ 1 ), . . . ,jcW, respectively, and it is zero otherwise. Since the n information sequences . . . ,jcW 
are coded independently, it follows that 




On the other hand, since the coded signals are sent over orthogonal AWGN channels, one has: 

P(Y\S) = f\p (y(*y*>) = f\f\p (yf ] \4 k) ) ■ (20) 
k=\ v J k=\i=\ v 7 

Taking into account the correlation model (0Q) of the information sequences, one obtains: 

p{X) = f[ P {x l ) (21) 

(=1 

where the generic term on the right-hand side of (I2TI) is given by ©. 

At this point, equation (fl"8T ) admits a Tanner graph representation and a corresponding belief 
propagation (BP) solution, provided that p(*W |^( fc )) (fc = 1, . . . ,n) can be expressed as a product 
of factors which depend on restricted subsets of all symbol variables. This is always possible if ^ 
(k = 1 , . . . , n) are convolutional codes or a serial or parallel concatenation of convolutional codes 
(i.e., turbo codes). Indeed, by introducing the state sequence as hidden variables, p(s^\x^) 
can be written as the product of factors representing the allowed state transitions [25]. Another 
situation where equation (fT8l) easily admits a Tanner graph-based representation is when ^ are 
LDPC systematic codes. In this case, p(s^\x^) can be obtained starting from N — L parity 
check equations, which involve a few parity and systematic bits, and the solution of (fl"8l) can be 
obtained through a graphical approach. 

Consider n separate Tanner graphs corresponding to the codes {^}^ =1 . A pictorial description 
of the global Tanner graph is shown in Fig. [71 where, for clarity, the variable nodes {jtjlflj are 

(k) 

explicitly shown. Each single variable node Xj (j = 1, . . . ,L, k = 1,. .. ,n) of the Tanner graph 
of &k is connected to the corresponding node Xj (j = 1 , . . . ,L, z ^ k) of the Tanner graph of 
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Tanner graph of C\ 



p(«l) 



i 1 ' b 




Tanner graph 
Of C n 




Tanner graph of c 2 



Fig. 7. Tanner graph for equation d!8t , 

^ through a connection node, marked by the joint pdf p(xj) in (|2l"l) . Note that this pdf depends 
on p. The connection nodes, upon receiving the logarithmic likelihood ratios (LLRs) messages 
from n — 1 of the n component Tanner graphs, send input LLRs to the other Tanner graph. The 

(£) 

LLR output by the connection node for the l-th component Tanner graph, denoted as LLR^ ut •, 
can be expressed as 



LLR 



(i) 

out,; 



In 



P [ xf = 



P x 



SO 



1 



In 



I P(*j) 



ln^ 



where j = 1 , . . . , L, In denotes the natural logarithm, and 

Xj Xj \ Xj ^X j , . . . ,Xj ,Xj 



(22) 



...,Xj 
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is the column vector denoting the bits at the input of the various sensor nodes, with the exception 
of the £-th one, at time epoch j. The factors {P(x'j)} denote the probabilities coming from the 
other n — 1 decoders (corresponding to the other sources). Assuming these n — \ outputs as 
independent, one can write 



^)=rp(*f)=n 



_w TTR w 



k=l £=11 -f e LLK inj 

where LLR,-„ ■ is the LLR associated with the /-th bit coming from the k-th decoder and x) is 
the logical negation of Xj. Therefore, (|22l) becomes: 



IP (xf = o^.) n ^-Jr 

*' V 7 ^ £=1 i +e LLR in.; 

LLR ou tJ - - ln — - ikj— m ■ w 



n x- LLR- 

e J m J 



Note that LLR- n ■ (k = 1, . . . ,n; k ^ £) may be seen as a priori information on the transmitted 
bits and can thus be easily taken into account by standard soft-input soft-output decoders. The 
conditional a posteriori probabilities P{xf = 0\x'j) and P{xj = l|jCj) in (|23l) can be computed 
by observing that, owing to the statistical characterization of the random variables {Zj} given 
at the beginning of Section HH we have: 

p(xf=0\x'fj = pP(b J = 0\x' j )+(l-p)P(b J = l\x' J ) 

P (xf = l\x'j) = (l-p)P(b J = 0\x' J )+pP(b J = l\x' j ) 

where bj has been defined in (Q])- The probability P(bj\x'j) can be computed as 

P(bj\x'j) ~ ^ ' 



where, according to (O, it is possible to write: 



p(x>) = I \p<(\-p) n - l -<+p n - l -<(l-p)< 
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n' h being the number of zeros of x'j. Moreover, from the model in (Q~|), it is straightforward to 
derive: 



p(bj = o,x>) = \p(z! ] =x l j ) = l -\p<{\- P y- l -< 



2 

P{bj = hx' j ) = Ip(*}=5$) = I p»- 1 -<(l-p)^ 



where 



(l) (e-i) (e+i) (n) 

7 ■ 7 ■ 7 ■ 7 ■ 



(i) (e-i) ue+i) Jn) 



Therefore, one can finally write: 



(1-P) 



n—l—ru, i j->w— 1— nl 



b(l- P 



LLRS tJ = ln- 



M 1+e m,J 



p' , b(l-p) n -"b+p"-"b(l-p)"b 



n #LLR.«. 
, 1 " LLR. W . 

k ~ l l+e m 'J 



(24) 



n 

k=] 



In [23], a simplified sub-optimal version of (|24j) . which takes into account pairwise a priori 
probabilities only, can be found. It is worth noting that the optimal combination rule (1241) and its 
sub-optimal (pairwise) version in [23] coincide in the case with n = 2. Therefore, (|24l) reduces to 
that showro in [22] for the case with n = 2. In Fig. [8l the LLR transformation (124)) is shown for 
n = 3 sources and p =0.95. Note that the LLR transformation (1241) is monotonic with respect 
to each input variable. 

The scheduling of the BP procedure on the overall graph can be serially performed as follows. 
We initialize the messages emitted by the function nodes {p{Xj)} L - =l to zero and run "internal" 
BP iterations within the component Tanner graph At the end of these BP iterations, messages 



Note that (10) in [22] contains a typographical error. 
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Fig. 8. LLR transformation J24b in a scenario with n = 3 sources and p = 0.95. 

{LLr[^}^ =1 are fed to the connecting nodes {p(xj)}j =1 which, in turn, emit new LLRs for 
the other component Tanner graph ^2. The same operation is repeated for the computation of 
the LLRs to be fed into ^3, and so on until The iterations between the n Tanner graphs, 
through the connection nodes, are referred to as "external." Note that the results in Section IVl are 
obtained using this BP scheduling. However, different scheduling can be considered, as in [18], 
where the BP procedure is performed on the overall Tanner graph without resorting to internal 
and external iterations. 

V. Performance Evaluation 

In the presence of SCCCs, we assume that the transmitters use identical rate- 1/2 linear codes, 
consisting of the cascade of an outer convolutional code (CC), a bit interleaver, and an inner 
CC [26]. This code structure has been shown to guarantee a very good performance in a classical 
AWGN scenario [26]-[28]. Moreover, as already observed in [16], our results confirm that, for 
balanced channels, the type of turbo-like channel code used at the sources is not critical, since 
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channel codes which perform well in the AWGN case without correlation are also good in the 
presence of correlation. On the other hand, in the unbalanced case, i.e., when the SNRs in 
the two channels are different, different channel coding strategies may entail better performance. 
Therefore, the design of good channel codes, which allow to approach the theoretical performance 
limits, is of interest. A possible example of good coding scheme was originally presented in [23] 
and is specified by the following generators: 



dinner (D) 



1+D 2 



1+D + D 2 + D 3 



'outer 



(D) 



1+D 2 1 



l+D + D 2 l+D + D 2 



It is worth noting that, unlike SCCCs optimized for transmission over a single AWGN chan- 
nel [26], this coding scheme requires the use of an outer recursive non-systematic CC. Moreover, 
note that the inner code has rate 1 and, therefore, no puncturing is needed. 

In the presence of LDPC codes, we assume that the transmitters use identical rate- 1/2 LDPC 
codes, each of them characterized by the degree distributions of the variable and check nodes, 
denoted as AldpcM an d PldpcMO An example of good LDPC code is that denoted as 
Irregular 3 in [23], with the following degree distributions: 

PldpcW = 0.00749jt 7 +0.99101jt 8 + 0.00150/ 

Aldpc (x) = 0.1 9606* + 0. 24039/ + 0.00228/ + 0.055 1 6/ + 0. 1 6602/ + 0.04088/ 
+0.01064x 9 + 0.00221/ 7 + 0.28636/ 9 . 

Note that this is a possible subset of pragmatic SCCCs and LDPC codes for orthogonal multiple 
access schemes with correlated sources. For instance, in [18], [19] the authors find optimized 
LDPC codes for a two-source scenario. In particular, the code proposed in [19] performs very 

4 This notation should not be confused with the previous usage of the mathematical symbols X and p. 
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well in all points of the feasible capacity region and is referred to as "universal." This code has, 
however, an encoding/decoding complexity larger than that of the codes analyzed in our work. 

The performance of the overall joint decoder can be analyzed by means of the exit chart method 
initially presented in [21], [22] for the two- source scenario. The extension of this method to the 
generic scenario with n > 2 sources is detailed in the Appendix. In the following, we present the 
feasible region for both SCCCs and LDPC codes. An exhaustive analysis of the EXIT curves 
for both channel codes is presented in [21], [22] and it is not reported here for conciseness. 

In Fig. |9[ Abai and A un b of the considered SCCC and LDPC code are shown, as functions on 
n, for p = 0.95. Theoretical limits are also reported as performance benchmarks. As one can 
see, while in the balanced case (%\ = fa = ■ ■ • = fa) both codes are limited to, for a given value 
of n, a performance far from the theoretical limit, the optimized SCCC extends significantly 
the feasible region, with respect to the LDPC, in the unbalanced scenario, i.e., when n — 1 of 
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the n channel SNRs are significantly higher than the remaining one. This is consistent with 
the results for n = 2 shown in [21], [22]. Moreover, for the LDPC code, Ab a i is very close to 
^unb for n = 2 and they coincide for n > 3, i.e., the feasible region is square for n > 3. As Abai 
(and Aunt,) remains constant for n > 5, the square feasible region of the LDPC code does not 
enlarge for increasing values of n. These observations lead to the conclusion that classical LDPC 
codes are not effective in the considered multiple access scenario and one should resort to more 
complicated structures, as proposed in [19]. Considering the SCCC, one can observe that the 
distance from Ab a i to \ n b seems to increase from n = 2 to n = 5, and tends to remain constant 
from n = 5 on. As the SCCC is closer to the theoretical limit, the obtained results confirm the 
conclusions in Subsection IIII-BL i.e., the fact that a good code in a two-source scenario is good 
also in scenarios with larger values of n. 

VI. Concluding Remarks 

In this paper, we have considered orthogonal multiple access schemes with a generic number 
of correlated sources, where the correlation is exploited at the AP. In particular, each source uses 
a classical channel code to transmit, through an AWGN channel, its information to the AP, where 
component decoders, associated with the sources, iteratively exchange soft information by taking 
into account the correlation. In the presence of n sources, we have first characterized the multi- 
dimensional feasible capacity region, defined as the ensemble of the channel parameter n-tuples 
where arbitrarily small probability of error is achievable. Our results show that, for asymptotically 
large values of the number of sources, the feasible region approaches a hyperoctant. Moreover, 
increasing the number of sources beyond 3 does not lead to significant performance improvement, 
as the projection of the feasible capacity region on a bidimensional hyperplane becomes only 
slightly larger. Therefore, a capacity-achieving channel code for a two-source scenario will likely 
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be a good code also in scenarios with larger values of the number of sources. Then, on the basis 
of an EXIT-based approach, we have computed the system multi-dimensional feasible region for 
two representative codes: an SCCC and an LDPC code. Our results with these practical coding 
schemes confirm the predictions of our information-theoretic framework. In particular, a good 
code for a two source scenario is expected to be good also in scenarios with larger values of n. 



In order to evaluate the performance of the overall joint decoder, we consider an EXIT- 
based approach. In particular, we refer to the EXIT-based approach proposed in [29] and further 
analyzed in [21] to characterize the internal behavior of concatenated and LDPC codes, i.e., the 
evolution of the LLRs within each component decoder. Without loss of generality, we focus on 
code %, i = l,...,n, and assume that the corresponding source transmits the all-zero sequence. 
Therefore, the LDPC decoder receives, at its input, a sequence of Gaussian observables specified 
by the channel SNR y. The channel LLRs are fed to the input of variable nodes. Density 
consistency is imposed by assuming that each LLR is modeled as a Gaussian random variable 
with mean /i ch and variance 2/i ch , i.e., with pdf |f 



Accordingly, y= jU^ h /2/i c h = /i c h/2. Using this assumption, the EXIT-based approach proposed 
in [29] allows to evaluate the SNR of the extrinsic information messages at the output of the 
component decoder. 

In Fig. [101 an illustrative scheme to analyze the evolution of the a priori information through the 
z'-th component decoder (i = 1 , . . . , n) is shown. To encompass the presence of a priori information 

5 The variable z should not be confused with the output of the BSCs in the correlation model Q}. 
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Fig. 10. Scheme for the analysis of the evolution of the a priori information. 



coming from the other component decoders (either SCCC or LDPC code decoders), let us denote 
by SNR>^, ke {l,...,n}; k^i, the SNR of external messages {LLRp}^ =1 entering the set 
of connection nodes denoted by P(X). We assume that also the messages {LLRy }y =1 have a 
Gaussian distribution with mean 2SNR^ and standard deviation J 4SNr|^, so that each pdf 

(k) 

is completely determined by a single parameter SNR-\ These messages are processed by the 
set of connection nodes {p(xj)} to produce a priori information messages {LLR^ t ^}y =1 for the 
variable nodes of the Tanner graph of %. 
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Denote the pdfs of the messages {LLR^ }^ =1 as a^ k \z) (k = l,...,n; k ^ i) and the pdf 
of {LLR 0U tj}j = i as ^out(z)- It is worth noting that an EXIT-based approach requires that 
all {o^ k \z)}1 =l are densities of messages corresponding to all-zero transmitted information 
sequences. Hence, taking into account the correlation model of {x^ k '}1 =l introduced in Section [II] 
(recall a common virtual originating bit), for analysis purposes it is necessary to introduce two 
consecutive BSC blocks, each with cross-over probability p, for each sequence jcW, i ^ k: at the 
output of the first one there is an estimate of the sequence whereas at the output of the 
second block there is an estimate of X{. Since a BSC with parameter p "flips" a bit at its input 
with probability p, the sign of the corresponding LLR is flipped with the same probability. 

The messages at the input of the second BSC (i.e., at the output of the first set of BSCs), 
denoted as {LLR'^}, are then characterized by the following pdfs: 

( $( z )=pa ( - k \z) + (l-p)a®(-z) k=l,...,n k^i. (25) 

The messages at the input of the set of connection nodes are denoted as {LLR^.}y =1 . Note that 
this scheme is compliant with that, relative to a scenario with n = 2, proposed in [21], [22]. In 
this case, in fact, the cascade of two BSC-like blocks with parameter p is equivalent to a unique 
BSC-like block with parameter pf = p 2 + (1 — p) 2 . 

The pdf &ou t (z) of {LLR^ t ^}^ =1 can eventually be computed according to (|24|) . with input 
messages {LLR^}^ =1 , by applying well-known results for pdf transformation [30]. Note that, 
unlike cS k \z) and T^'J, b^(z) cannot be Gaussian. It can be easily seen that the analytical 
computation of ^out(^) nas an exponential complexity on the order of A^ 1 , where N p df is the 
number of samples of the numerical representation of the pdfs used in the computer solver. In 
fact, for each output sample in bgl t (z), all possible combinations (of length n— 1) from the input 
pdfs {a^(z)} which give this sample should be analyzed. In order to make the computational 
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complexity feasible, we resort to simulations to compute the distribution b^l t (z) at the input 
of the decoder. A closed-form expression for b^l t (z) is provided in [22] for n = 2 and can be 
extended to scenarios with a generic number of n. 

After a fixed number of internal message passing decoding operations^ the extrinsic informa- 
tion sequence is extracted from the soft-output information sequence at the output of the decoder 
and the output SNR, denoted as SNR^] t , is evaluated. For a fixed value of the channel SNR, the 
above steps allow to numerically determine the ^-dimensional function Z such that: 

SNR^t = Z (sNR m } , . . . , SNr£~ 1} , SNr£ +1) , . . . , SNR^ , y) = Z(SNR in , y) (26) 

where SNR in = (SNR m \...,SNR m ~ 1 \SNR m +1 \...,SNR^ ) ). In particular, Z represents the 
input-output characteristic function between the input a priori information from the other n — 1 
decoders and the information output by the single channel decoder under analysis. 

As previously shown, the component decoder can now be analyzed through a classical density 
evolution approach [31], the only difference being the fact that the messages at the input of the 
decoder associated with the information bits need to be modified in order to model the presence 
of a priori information. In particular, in the iterative decoding procedure the a priori information 
from the other decoder is added to the channel information at the input of the information bits 
of the decoder. From the message density viewpoint, this corresponds to convolving the a priori 
message pdf &out( z ) by the Gaussian channel message pdf r cn (z): 

m W (z)=r ch (z)®^ t (z) (27) 
where eg) denotes the convolution operator. However, one should note that this operation is 

In our numerical results, the number of internal iterations was set to 50 for LDPC-coded scenarios. In the presence of a 
SCCC, instead, the number of internal iterations between convolutional decoders employing the BCJR algorithm was set to 10. 
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done only in correspondence with the information bits. For instance, in the presence of LDPC 
decoding the convolution operation (ITTT) is statistically done over a fraction, equal to the code 
rate r, of the input variable nodes and the remaining variable nodes have, at their inputs, only 
the pdf r c h(z). At this point, the density evolution procedure can be implemented in the classical 
way, by iterating the concatenated decoder or the sum-product algorithm for a fixed number of 
iterations. Note that the pdf at the input of the decoder is no longer exactly Gaussian, due to 
the transformation (1241) . However, the shape of m^>{z) is similar to that of a Gaussian pdf (see, 
e.g., [22]) and, therefore, one can conclude that the proposed EXIT approach is still accurate, 
although not exact. Numerical results, not reported here for lack of space, confirm this statement. 

The values of the characteristic points Abai and A un b can be obtained from the EXIT surface 
(|26l) in the following way. In the unbalanced case, the value of the asymptote of the feasible 
region can be computed by, first, assuming that the a priori information sequences coming 
from the other decoders are characterized by sufficiently large SNRs, assumed to be equal, i.e., 
SNR^ = SNR in > (k = 1, . . . k ^ i), and, then, finding the value of X (or, equivalently, y) 
for which there is decoding convergence, e.g., SNR 0U t SNR; n . 

In the balanced case, the n channels are characterized by the same SNR y = % a \ (balanced 
channels). We can thus analyze the joint decoding convergence by drawing, for a given value of 
y, the Z hypersurface and its inverse Z _1 . Moreover, since the decoder is working in a region 
characterized by the same SNR for all channels, it is reasonable to assume the same a priori SNR 
from all the sources, i.e., SNR-^ = SNR; n (k = 1 , . . . , n; k ^ i). Therefore, the analysis, carried out 
in [21], [22] for two sources, can be applied here, by considering the curve Zb a i =Z(SNRi n ,y) 
and characterizing the decoding convergence during external iterations (between the component 
decoders associated with the n sources): the farther the curves, the faster the joint decoding 
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convergence to (theoretically) zero BER. When the two curves touch each other, then global 
decoding convergence is not achieved and the BER is bounded away from zero. The value of 
Tbal (or, equivalently, of Abai) is the minimum for which convergence is guaranteed. 
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